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Abstract 

Four Introduction to Psychology and 6 Developmental Psychology Graduate Teaching Assistants 
collected student ratings from their students with the newly created Evaluation of College Instructor 
(ECI) rating instrument. A total of 82 Introduction to Psychology students and 186 Developmental 
Psychology students completed the ECI shortly before the end of the Spring 2001 semester. The 
data for all Intro and all Developmental sections were combined, and then analyzed separately. 
Results revealed that a different set of research variables influenced student ratings in the two types 
of courses. Specifically, whether the class was Challenging and whether the students were 
Motivated by the Instructor to perform well in the course, were the most (and the only) potent 
variables for the Introduction to Psychology students. Whether students were Motivated by the 
Instructor, and whether students Struggled to Understand the material influenced the ratings of the 
Developmental Psychology students. Most interestingly, when six different dimensions of Teaching 
Performance were regressed onto the ten research variables, the same predictors emerged as 
significant predictors of student ratings for those specific dimensions. 
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Empirical Examination of Factors That Influence Student Ratings of Instructors 

It is a common practice for colleges and universities to involve their clients - the students - 
in the evaluation of their instructors (Bedggood & Pollard, 1999). This practice has been disputed 
since its beginning, mostly because many believe that it directly threatens the academic freedom 
instructors are supposed to enjoy (e.g., McKeachie, 1997). Despite these protests, student 
evaluations of instructors are used for administrative decisions (e.g., promotion, tenure securement, 
and even termination) concerning those instructors. Therefore, it is of little wonder that student 
evaluations have received considerable research attention, and that there still is wide disagreement 
among teachers and researchers alike concerning whether student evaluations should (or even 
could) be used as an accurate measure of instructor performance. 

To make things even more complicated, a host of intervening variables that influence how 
students rate their instructors exists. Among those variables most often studied by researchers are 
• (a) class standing, (b) expected grade, (c) sex of the student and the instructor, (d) years of 
experience of the instructor, (e) overall Grade Point Average (GPA), etc. Generally, these variables 
fall within the categories of student characteristics, course characteristics, and instructor 
characteristics. The goal of this study was to examine which of the student, course, and instructor 
characteristics influence students ratings of their instructors. 

Student Characteristics 

Among the more popular student characteristics studied by previous researchers are class 
standing, expected grade, overall GPA, and prior subject interest. However, many of the findings 
contradict one another. For instance, class standing generally is expected to affect student ratings of 
instructors. Still, in her analysis of a multitude of variables influencing the validity of student 




4 



Empirical Examination 4 



ratings, Langbein (1994) found that upper level students discriminated more, or were more critical, 
in their evaluations of courses, but not of instructors. 

A similar discrepancy exists in findings of the effects of expected grades. On the one hand, 
Langbein (1994) reported that students who expected higher grades gave higher ratings to their 
instructors. Because of this and similar findings (e.g., Greenwald & Gillmore, 1997), d’Apollonia 
and Abrami (1997) some have argued that faculty may purposefully give students higher grades to 
receive higher evaluations. On the other hand, however. Marsh (1982) determined that the effect of 
expected grade was explained in part by prior subject interest. Students who indicated interest in the 
course rated the course higher and indicated that they expected higher grades. 

Students with higher GPAs are also thought to better discriminate in their evaluations of 
teachers and courses, although the exact direction of this relation is not very clear. For example, in 
Langbein’s study (1994), there was a strong negative correlation between GPA and ratings, 
implying that better students are more disapproving of their courses and instructors, and possibly . 
more discriminating in their judgments. However, Marsh (1982) failed to demonstrate a relation 
between GPA and overall rating. 

One variable that seems to be free of controversy is prior subject interest. Marsh (1982) 
showed that prior subject interest was a very important moderating variable. Specifically, students 
who indicated prior interest in the course material rated the course and the instructor higher. . 
Therefore, more interested students perceive teaching as more effective. 

Course Characteristics 

Many course characteristics have been studied by previous researchers, among which 
required courses, course size, and high workload — high difficulty courses. Again, many of the 
findings contradict one another. For instance, although the expectation is that required courses 
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would be perceived as a burden, and subsequently rated lower than elective courses, this relation 
could not be demonstrated by Langbein’s (1994) study, required versus elective courses were rated 
in a similar manner. However, Marsh (1982) did show some minor differences between students’ 
ratings of required versus elective courses. 

Research has shown that class size is related to student ratings of course quality but not of 
instructor quality. For instance, Langbein’s (1994) research revealed that students in smaller classes 
provided high ratings but students in larger classes also rated the instructors high, just as students in 
smaller classes did. In other words, instructional quality was not seen as decreasing as the class size 
increased, but course quality was (because the ratings for course decreased). However, others have 
concluded that smaller classes tend to provide better ratings (e.g., Greenwald & Gillmore, 1997; 
McKeachie, 1997), so the research is unclear. 

Although it might seem counter-intuitive. Marsh (1982) reported that courses that were 
harder and required more time outside of class (i.e., high workload — high difficulty courses) 
received higher ratings. Indeed in his study, prior subject interest and difficulty level were the two 
most influential variables in terms of explaining student ratings. This result again seems to confirm 
that student interest and challenge is of utmost importance. 

Instructor Characteristics 

Researchers also have examined a variety of instructor characteristics. The two most popular 
characteristics are experience and sex of the instructor. In terms of experience, Langbein (1994) 
found that instructor evaluations become more positive during the first 13 years of teaching but that 
this pattern switches in the mid-teen years of experience when bum-out or age turn student 
evaluations in a negative direction. 
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In terms of instructor sex, Langbein (1994) found that female faculty members were treated 
differently than male faculty members. When a student expected a low grade from a female faculty 
member, this evaluation was regarded as “nonnurturing” and the female faculty received lower 
evaluations than her comparable male colleague. Thus, sex was shown to interact with expected 
grade. Centra and Gaubatz (2000) also examined sex interaction in the classroom. They found that 
female students consistently awarded higher rating to female instructors, whereas male students 
rated female instructors lower. However, there were no differences when male and female students 
rated male instructors. 

Present Study 

The research literature paints a complicated picture of the factors that affect student ratings 
of college instructors. Despite the multitude of studies examining the influence of numerous 
extraneous variables on student ratings, a more comprehensive picture is still to be drawn. The 
problem seems to be that different variables have been examined in the different studies, whereas 
researchers need to conduct studies that examine the most influential factors simultaneously. 

Therefore, the goal of this research was to determine how eight of the most frequently 
researched variables influenced students ratings in a large sample of Introductory Psychology and 
Developmental Psychology students. Specifically, these variables included, from the student 
characteristics. Class Standing, Expected Grade, GPA, Prior Subject Interest, and Student Sex, and, 
from the course characteristics. Required versus Elective Course, and Course Difficulty, 
operationalized here as two distinct variables - Amount of Effort Required and Challenging Course. 
In addition, two other variables were introduced: students’ level of Motivation by Instructor, and 
whether students Struggled to Understand concepts presented in class. Some of the other frequently 
examined by researchers variables could not be studied here because of sample limitations. For 
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instance, all classes were of similar size, all instructors had 1 to 2 years of experience, and all were 
women. 

Thus, these 10 variables served as the Research Variables in the study. It was hypothesized 
that they would influence the Overall Instructor Rating provided by the students. Further, it was also 
expected that these research variables might affect some of the dimensions of Teaching 
Performance (described below), but due to the lack of literature on how these variables might 
influence the different teaching performance dimensions, exact hypotheses could not be advanced. 

Method 

Participants 

A sample of 72 Introduction to Psychology (INTRO) students (50 women, 28 men), and 183 
Developmental Psychology (DEV) students (143 women, 40 men) completed the Evaluation of 
College Instructor (ECI) rating tool. Of those students who reported their GPA, 40 INTRO students 
and 68 DEV students reported a GPA of 2.10-3.00, and 35 INTRO students and 1 13 DEV students 
reported a GPA of 3.10-4.00. Further, 68 INTRO students and 132 DEV students indicated that the 
course was required for them to take, whereas 13 INTRO and 16 DEV students indicated that the 
course was an elective. Among the INTRO students, there were 7 seniors, 1 1 juniors, 14 
sophomores and 50 freshmen, whereas among the DEV students, there were 9 seniors, 25 juniors, 

51 sophomores and 98 freshmen. Lastly, in the INTRO sections, 32 students expected an A, 36 
expected a B, 1 1 expected a C and 2 expected a D. In the DEV sections, 97 expected an A, 70 
expected a B, 18 expected a C, and no one expected a D for the course. 

Procedure 

Four INTRO and 6 DEV Graduate Teaching Assistants distributed the Evaluation of College 
Instructor (ECI) rating instrument to their classes at the end of Spring 2001 semester. Students were 
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informed that this rating would not be used officially by the school, but were asked to be as honest 
as they would be on official university instructor evaluation forms. Further, all students were 
assured of the complete confidentiality of their ratings. 

Rating Instrument 

The Evaluation of College Instructor (ECI) rating scale was used in this research (Appendix 
1). The scale consisted of 5 demographic questions (GPA, expected grade, reason for taking the 
course, class standing, and sex), 25 Rate-Your-Instructor questions, where students provided ratings 
on a scale of 1 to 5 (1 = Never, 5 = Always), and 5 Rate-Yourself questions, where students 
provided ratings on a scale of 1 to 5 (1 = Very Low, 5 = Very High). 

The 25 Rate-Your-Instructor questions were organized in the following dimensions: 
LeamingA^alue (questions 1 - 3), Motivation of Student Learning (questions 4 - 5), Individual 
Rapport (questions 6 - 8), Organization/Clarity (questions 9-15), Examination/Grading Practices 
(questions 16 - 21), and Presentational Style (questions 22 - 25). The 5 Rate-Yourself questions 
asked students to rate their interest in taking.the course before they enrolled, the amount of effort 
the course required, the extent to which the course was challenging, the extent to which students 
were motivated by their instructor, and the extent to which they struggled to understand the 
material. 

The 5 demographic questions and the 5 Rate-Yourself questions were generated after a 
literature review of variables that influence student ratings. The 25 Rate-Your-Instructor items also 
were generated after a careful literature review, as well as a review of existing instructor rating 
scales. Specifically, a pool of 68 items was created, duplicate and near-duplicate items were 
eliminated, and the 25 final items, grouped within the six dimensions, were produced. This 
approach to constructing a rating scale, known as content analysis, is consistent with literature 
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suggestions (e.g., Centra & Gaubatz, 2000; Langbein, 1994; McKeachie, 1997; Sheehan & DuPrey, 
1999; Young & Shaw, 1999). 

Results 

The data were analyzed by class section first (not reported here); then all INTRO and all 
DEV sections were combined and analyzed together. Results revealed that a different set of research 
variables influenced the overall instructor ratings in the INTRO sections and in the DEV sections. 
Further, different variables affected the six dimensions differently in the two types of classes. Thus, 
the data from the Intro and Developmental sections are presented separately. 

Introduction to Psychology Sections 

Table 1 shows the correlations between the dependent variable of interest (Overall Rating) 
and the 10 predictor variables. As evidenced from the table, five out of the ten predictors were 
significantly correlated with Overall Rating. However, when these predictor variables were entered 
into a Linear Regression (see Table 2), only Challenging Course and Motivation by Instructor 
emerged as significant predictors {fi= 0.40, p = 0.003; P= 0.49, p < 0.001 respectively). Those two 
predictors, when controlling for the influence of the other eight, managed to explain 36% of the 
variance (adjusted R^). A re-examination of the zero-order correlations reveals that Motivation by 
Instructor was indeed the most highly correlated variable with Overall Rating. Therefore, when ten 
different student and course characteristics shown by previous research to influence students’ 
ratings are considered in tandem, how much the instructor motivates his/her students, as well as 
how much he/she challenges them, appear to be of utmost importance. 

Table 2 also shows the beta regression coefficients obtained when the six dimensions of 
Teaching Performance were regressed onto the ten predictors. Intriguingly, the same two predictors 
emerged as significant when each of the teaching performance dimensions, were regressed on the 
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predictor variables. Specifically, how much the instructor motivated the students influenced the 
students’ ratings of LeamingA^alue, Motivation of Students, Organization/Clarity and 
Presentational Style, and how much the instructor challenged the students influence their ratings on 
Individual Rapport and Examination/Grading. This combination of predictors managed to explain 
an average of 26% of the variance in Overall Student Rating. Finally, student GPA also emerged as 
a significant predictor of Motivation of Students, and how much the students struggled to 
understand concepts presented in class affected the Examination/Grading dimension. 

Developmental Psychology Sections 

Table 3 shows the correlations between the dependent variable of interest (Overall Rating) 
and the 10 predictor variables. As evidenced from the table, six out of the ten predictors were 
significantly correlated with Overall Rating. However, when these predictor variables were entered 
into a Linear Regression (see Table 4), only Motivation by Instructor and Struggled to Understand 
emerged as significant predictors {fi= 0.40,/? < 0.001; -0.19,/? = 0.10 respectively). Those two 

predictors, when controlling for the influence of the other eight, managed to explain 28% of the 
variance (adjusted R^). A re-examination of the zero-order correlations reveals that Motivation by 
Instructor was indeed the most highly correlated variable with Overall Rating. Therefore, when ten 
different student and course characteristics shown by previous research to influence students’ 
ratings are considered in tandem, how much the instructor motivates his/her students, as well as 
how much the instructor is able to present concepts so that student do not struggle understanding 
them, appear to be of utmost importance. 

Table 4 also shows the beta regression coefficients obtained when the six dimensions of 
Teaching Performance were regressed onto the ten predictors. This combination of predictors 
managed to explain an average of 20% of the variance in Overall Student Rating. It should be noted. 
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however, that only 7% of the variance in Presentational style was explained by the predictor 
variables. Intriguingly, how much the instructor Motivated the students influenced all six 
dimensions of teaching performance. Struggled to Understand influenced LeamingA^ alue and 
Examination/Grading, Prior Interest in the Course affected LeamingA^ alue, and lastly. Effort 
Required affected the LeamingA^alue dimension. Therefore, Motivation by Instructor again proved 
to be the most influential factor. 

Discussion 

Introduction to Psychology Cumulative Results 

Clearly, the only two variables that influenced overall ratings of instructors were 
Challenging Course and Motivation by Instructor. Those two predictors, when controlling for the 
other eight, accounted for a respectable 36% of the variance (adjusted). To see where the difference 
lied more precisely, two one-way ANOVAs were performed with Overall Rating serving as the DV 
and Challenging Course and Motivation by Instructor serving as the IVs. The first ANOVA 
(Overall Rating x Challenging Course) was significant (F( 4 , 77 ) = 6.09, p < 0.001), and a Scheffe 
comparison revealed that those students who rated the course as Very Highly Challenging (M = 

2.69, SD = 1.38) rated the course significantly lower than students who thought the course was Very 
Challenging (M= 4.44, SD = 0.33), Medium Challenging (M= 4.23, SD = 0.43) and Low 
Challenging (M= 3.92, SD = 0.86). 

Recall that course difficulty was operationalized as two distinct variables - amount of effort 
required and challenging course. Also recall that Marsh (1982) concluded that high workload high 
difficulty courses are rated higher by students than low difficulty courses. Clearly, then, these 
results contradict Marsh’s conclusions. However, when interpreting this discrepancy, the 
characteristics of the sample must be taken into consideration. Fifty of the Intro students were 
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freshmen, whereas only 32 were classified as sophomores, juniors and seniors combined. Thus, 
considering that the majority of the Intro to Psychology students was freshmen, it is not surprising 
that Challenging Course was one of the two variables of import. It might be that freshmen are 
unprepared to deal with the difficulties and the challenges of college life, and that, subsequently, 
they punish their instructors by awarding them lower ratings. Thus, class standing is probably a 
modifier in the relationship between how challenging the course is perceived to be and the overall 
instructor rating. The contentions that Class Standing and Challenging Course are related finds 
support in Table 1, which shows a significant zero-order correlation between the two variables. 

The second variable of import. Motivation by Instructor, clearly was the variable of utmost 
significance. A one-way ANOVA with this variable used as an IV was of course significant, 77 ) 
= 17.30,/? < 0.001), and a Scheffe comparison revealed that students who rated their Motivation by 
Instructor as Low (M= 2.98, SD =1.16) rated the instructor significantly lower than students who 
rated their Motivation by Instructor as Medium (M= 4.02, SD = 0.41), High (M= 4.31, SD = 0.39) 
and Very High (M = 4.57, SD = 0.25). 

Two issues arising from these results deserve mention. First, the difference between low and 
high overall ratings lies in whether the students reported their level of motivation by the instructor 
as low, or as medium, high and very high. In other words, as long as students were at least 
somewhat motivated by their instructors to do well in the course, they awarded higher ratings. 
Second, this variable is of high interest itself Student motivation has been repeatedly studied by 
researchers, and has repeatedly been reported to be an important determinant of student rating (e.g.. 
Marsh, 1982). However, students’ level of motivation bv instructor has not enjoyed much research 
attention, with the exception of Young and Shaw’s (1999) article where ineffective teachers were 
viewed as those who did not motivate their students to learn and where motivating students to do 
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their best was the fifth (out of six) variables that accounted for 87% of the variability in Young and 
Shaw’s criterion of teaching effectiveness. Still, in the present study, this variable is the most potent 
of all. What must be asked, then, is what exactly does “motivation by instructor” mean. Maybe this 
variable taps into rapport with instructor, that is, if students feel they are getting personalized 
attention and encouragement, then they feel their instructor is trying to motivate them, or help them, 
do better in the course. Clearly, this finding alone pleas for further research. 

Developmental Psychology Cumulative Results 

Here, two variables were of interest again —the familiar Motivation by Instructor, along with 
Struggled to Understand (concepts presented in class). As with the INTRO data, two one-way 
ANOVAs were performed in order to reveal which groups of students rated instructors lower or 
higher on their teaching performance. When Struggled to Understand (concepts presented in class) 
was used as the IV (and Overall Rating - as the DV), the F-test was significant (F( 4 , isi) = 4.69,/? = 
0.001), and Scheffe’s comparison revealed that those students who reported their level of Struggled 
to Understand as Very Low (M = 4.5\, SD = 0.33) rated the instructor significantly higher than 
students whose level on this variable was Low (M= 4.21, SD = 0.38), Medium (M = 4.21, SD = 
0.48), High (M= 4.14, SD = 0.57) and Very High (M= 4.14, SD = 0.43). 

Considering that Developmental is a more specialized class than Intro, one likely 
interpretation of this finding might be that students view the subject more seriously and strive to 
understand the material. Consequently, they are disappointed in the instructor when that material is 
not made easy for them to understand. As a result, they lower their ratings of their instructors’ 
teaching performance. Interestingly, class standing does not seem to be responsible for this result 
because freshman students were in the majority in this dataset as well. This is also confirmed by the 
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lack of a significant zero-order correlation between Class Standing and Struggled to Understand in 
Table 3. 

Motivation by Instructor came in as the most potent variable in this sample as well. A one- 
way ANOVA yielded significance (F( 3 , i 82 ) = 20.25,/? < 0.001), and Scheffe’s post hoc test revealed 
that students who rated their Motivation By Instructor as Low {M = 3.95, SD = 0.61) and Medium 
(M= 3.98, SD = 0.38) gave the instructors ratings significantly lower from the ratings of students 

who rated their Motivation by Instructor as High (M= 4.32, SD = 0.39) and Very High (M= 4.61, 

0 

SD = 0.29). 

An interesting difference in how this variable affected Overall Instructor Rating between the 
INTRO and the DEV sections emerged. In the INTRO sections, as long as students were at least 
somewhat (i.e.. Medium and higher) motivated by their instructors, their overall ratings were high. 
Only the students who reported their level of Motivation by Instructor as Low did not give high 
ratings. However, in the DEV sections, students had to be Highly or Very Highly motivated by their 
instructors in order to give high ratings. Being motivated at a Medium level was not good enough, 
not to even mention Low levels of Motivation by Instructor. What this probably tells us is that 
Developmental students tend to be more demanding of their instructors than Intro students. The 
unanswered question then is. Why is Motivation even more important in this data set considering 
that the majority of the students in both groups were freshmen. More importantly, however, what 
truly stands behind “Motivation by Instructor” needs to be investigated further. 

Dimensions of Teaching Effectiveness Results 

One of the more important contributions of the present study is that it not only examined the 
influence of a combination of variables on overall instructor rating, but also on different dimensions 
which have been said to make up teaching performance (often referred to in the literature as 




15 



Empirical Examination 15 



“teaching effectiveness”)- A finding consistent with the major results from the analyses where 
Overall Rating was the DV, is that the same predictors still held in the analyses where the teaching 
performance dimensions were the DVs. Arguably, however, the more interesting finding in terms of 
what affects the dimensions of teaching performance, is that there was no uniformity between the 
two data sets - what predicted some of the dimensions in the Intro data did not necessarily predict 
the dimensions in the Developmental data. Further, the Developmental data seemed to be more 
susceptible to influences than the Intro data. This possibly indicates that a different set of dynamics 
is at work in Intro to Psychology and in Developmental Psychology, which, consequently, means 
that student ratings in Intro classes should not be treated the same as student ratings in 
Developmental classes. 

Taking this argument one step further, it might even be held that different classes, regardless 
of whether they belong to the same or to a different discipline, should be evaluated with rating 
scales specifically tailored to the classes. For again, what seems to be important to Intro students 
does not necessarily seem to be important to Developmental students and vice versa. In other words, 
as Bedggood and Pollard (1999) stated in their article, and this research lends further support to 
their argument, is that “a standard evaluation form will not be appropriate for all subjects, for all 
courses, or for all fields of study” (p. 134). 

Limitations and conclusions 

The present study utilized a newly-created Instructor Rating Scale. Although all of the items 
on the rating scale were incorporated from the literature and from existing scales, the ECI itself has 
not been validated. Further, it is possible that students may not have taken the task of rating their 
instructors seriously because they were informed that their ratings will not be used for 
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administrative purposes. However, there were absolutely no indications that this might have been 
the case. 

These possibilities notwithstanding, three points of import must be made here. First, 
Motivation by Instructor was the only variable that affected both the Intro and the Developmental 
overall instructor ratings. Further, it accounted for big portions of the variance in overall instructor 
rating in both samples. Thus, it is somewhat puzzling that this variable has not been studied by 
researchers more. Understanding what being motivated by one’s instructor truly means is important, 
because this could help teachers improve their quality of teaching, and their ratings. 

Second, it is also interesting to point out that the other two variables affecting Overall 
Instructor Rating differed for the Intro and the Developmental classes. The only logical explanation 
of this might be that the dynamics of the two classes are simply too different, and that therefore 
teacher effectiveness may constitute different things in the eyes of the Intro students and in the eyes 
of the Developmental students. This raises a bigger problem, however. Could this not mean that 
student ratings would be influenced by different variables in every single course. In general, Intro 
and Developmental are not considered to be widely different - both are from the Psychology 
curriculum, both are taken mostly in the freshman or sophomore years, and, in the present sample, 
both are taught by female GTAs with one-to-two years of experience. Therefore, how can we be 
sure that studies that summarize student ratings among various courses, and even among various 
disciplines, actually report the true state of affairs? In this respect, the present study raises more and 
newer questions that need to be answered before we could confidently state that we understand what 
stands behind student ratings. 

Lastly, a third point that must be commented on, is that many of the variables examined in 
previous studies, failed to affect Overall Instructor Rating in this study. However, some of those 
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variables affected some of the six different dimensions. This raises yet another question: should we 
be satisfied with knowing what affects overall instructor ratings, and ignore what affects the 
different components of teaching performance? Should we also not know what affects students’ 
perceptions of Individual Rapport, or of Grading Practices, or of Learning, to name just a few? It 
might be time to look more into the details, and less into the overall picture, in order to truly 
understand what affects teaching performance and its components. 
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Table 1 

Means, Standard Deviations and Correlations between the study variables for the INTRO data 



Variable 


M 


SD 


1 


2 


3 


4 


5 


6 


7 


8 


9 


Overall 


4.17 


0.57 




















rating (1) 
GPA (2) 


3.43 


.55 


.25* 



















Expected 
Grade (3) 


4.20 


.81 


.21* 


.60** 


— 














Reason (4) 


.84 


.37 


-.03 


-.13 


-.14 


— 












Class 


1.69 


1.00 


.01 


-.04 


.01 


.20* 












Standing (5) 
Sex (6) 


.36 


.48 


-.16 


.03 


-.08 


-.04 


.21* 











Prior 


3.01 


1.06 


.20* 


.17 


.19 


-.19 


-.18 


-.19 








Interest (7) 
Effort 


3.33 


.82 


-.12 


-.33** 


-.41** 


-.04 


-.15 


-.25* 


.16 






Required (8) 
Challenging 

(9) 


3.01 


.75 


.23* 


- 33** 


-.40** 


.03 


-.25* 


-.21* 


.07 


.43** 


— 


Motivated 


3.71 


.81 


.53** 


-.02 


.11 


.02 


-.17 


- 33** 


.36** 


.16 


.27* 


(10) 

Struggled 

(11) 


2.40 


.93 


-.12 


-31** 


-.36** 


.26* 


-.10 


-.14 


-.11 


35** 


.52** 





Note. Correlations between these variables and the individual teaching performance dimensions are not shown. 
Variable coding scheme is provided in Appendix 1 . All tests are two-tailed. 

*p<0.05; **p<0.01 
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Table 2 

Regression of Overall Rating and the Six Teaching Performance Dimensions on the Research Variables for the INTRO 
data 



Predictors 


Overall 

Rating 


D1 


D2 


D3 


D4 


D5 


D6 


GPA 


.23 


.17 


.28* 


.15 


.14 


.23 


.18 


Expected 


.03 


.21 


.13 


.11 


.02 


-.04 


-•13 


Grade 

Reason 


-.01 


.05 


.03 


-.08 


.02 


-.02 


-.03 


Class 


.16 


-.02 


.05 


.15 


.14 


.22 


.14 


Standing 

Sex 


-.03 


.04 


.03 


.01 


-.03 


-.02 


-.03 


Prior 


-.02 


.02 


-.01 


.03 


-.04 


-.01 


-.05 


Interest 

Effort 


-.18 


.06 


.10 


-.22 


-.24 


-.16 


-.22 


Required 

Challenging 


.40** 


.21 


.08 


.55** 


.34* 


.46** 


.19 


Motivated 






.58** 


.10 


.43** 


.31* 


.50** 


Struggled 


-.20 


-.08 


.13 


-.25 


-.16 


-.28* 


-.09 


F value 


5.09* 


4.54* 


5.58* 


2.44 


3.14* 


2.97* 


2.66 


R^ 


0.45 


0.42 


0.47 


0.28 


0.34 


0.32 


0.30 


R^adj 


0.36 


0.33 


0.39 


0.17 


0.23 


0.22 


0.19 


df 


(10,62) 


(10, 62) 


(10, 62) 


(10, 62) 


(10, 62) 


(10, 62) 


(10, 62) 



Note. D1 = LeamingA^alue; D2 = Motivation of students, D3 = Individual rapport, D4 = Organization/Clarity, D5 = 
Examination/Grading, D6 = Presentational style. Cell entries are standardized confidents. All significance tests are two- 
tailed. 

*p<0.05;**p<0.01 
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Table 3 

Means. Standard Deviations, and Correlations between the study variables for the DEV data 



Variable 


M 


SD 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 11 


Overall 


4.26 


.44 






















rating (1) 
GPA (2) 


3.59 


.54 


.08 





















Expected 
Grade (3) 


4.43 


.66 


.21** 


.49** 


— 
















Reason (4) 


.88 


.33 


-.05 


.11 


-.02 
















Class 


1.70 


.88 


-.02 


-.05 


.03 


-.19** 














Standing (5) 
Sex (6) 


.22 


.41 


1 

00 

* 

* 


.03 


-.08 


.003 


.23** 













Prior 


3.39 


.90 


.21** 


.02 


.19* 


-.17* 


-.02 


-.19** 










Interest (7) 
Effort 

Required (8) 


3.47 


.64 


.18** 


.02 


-.01 


.08 


.04 


.06 


.09 


.65** 






Challenging 

(9) 

Motivated 


3.26 


.70 


.07 


.02 


-.12* 


-.03 


.05 


.10 


.03 


— 




3.83 


.76 


.50** 


.13* 


.22** 


.004 


-.03 


-.15* 


.12* 


.26** 


.21** 




(10) 

Struggled 

(11) 


2.28 


.94 


-.26** 


-.22** 


-.30** 


-.06 


.08 


.11 


-.09 


.27** 


.38** 


-.18** _ 





Note. Correlations between these variables and the individual dimensions are not shown. All tests are two-tailed. 
* p<0.05; ** p<0.01 
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Table 4 

Regression of Overall Rating and the Six Teaching Performance Dimensions on the Research Variables for the DEV 
data 



Predictors 


Overall 

Rating 


D1 


D2 


D3 


D4 


D5 


D6 


GPA 


-.04 


-.14 


-.03 


.07 


-.04 


-.01 


-.04 


Expected 


.06 


.05 


-.02 


.003 


.15 


-.01 


.06 


Grade 

Reason 


-.06 


-.02 


-.09 


-.02 


-.04 


-.05 


-.06 


Class 


.01 


-.01 


-.03 


-.09 


.02 


.05 


.07 


Standing 

Sex 


-.09 


-.09 


-.04 


-.12 


-.09 


-.08 


.02 


Prior 


.09 


.15* 


.03 


.009 


.12 


.06 


.05 


Interest 

Effort 


.15 


.19* 


.09 


.08 


.15 


.12 


.07 


Required 

Challenging 


-.03 


.03 


.07 


-.15 


-.03 


-.07 


.06 


Motivated 


.40* * 


.35** 


.58** 




.24** 


29** 


.24** 


Struggled 


-.19* 


-.18* 


-.02 


-.12 


-.13 


-.26** 


-.12 


F value 


7.98** 


7.66** 


11.99** 


4.49** 


4.47 


4.74** 


2.39* 


R^ 


0.32 


0.31 


0.41 


0.21 


0.21 


0.22 


0.12 




0.28 


0.27 


0.38 


0.16 


0.16 


0.17 


0.07 


df 


(10, 170) 


(10, 170) 


(10, 170) 


(10,170) 


(10, 170) 


(10, 170) 


(10, 170) 



Note. Dimension 1 = Learning alue; Dimension 2 - Motivation of students, Dimension 3 = Individual rapport, 
Dimension 4 = Organization/Clarity, Dimension 5 = Examination/ Grading, Dimension 6 = Presentational style 
Cell entries are standardized confidents. All significance tests are two-tailed. 

* p< 0.05; **p< 0.01 
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Appendix 1 

The Evaluation of College Instructor rating scale 
Section 1 - Demographic Information 

1. What is your Grade Point Average? 

0.00-1.00(1) 1.10-2.00(2) 2.10-3.00(3) 

NA (0) 

2. What grade do you expect to receive from this course? 

F(l) D(2) C(3) B(4) A (5) 

3. Which of the following best describes your reasons for taking this course? 

Required (1) Elective or Other (0) 

4. Which best describes your class standing? 

Freshman (1) Sophomore (2) Junior (3) 

Graduate (5) 

5. What is your sex? 

Male (1) Female (0) 

Section 2 - Rate you Instructor 

Please indicate how often your instructor. . . 

1 . increased your appreciate for the course material 

2. improved your understanding of concepts in the field 

3. achieved the course outcomes as stated in the syllabus 

4. encouraged students to engage in self-initiated learning 

5. motivated you to do high quality work 




.10-4.00 (4) 



Senior (4) 
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6. was accessible for help outside of class 

7. was condescending (i.e., talked down) to students (Reverse Scored) 

8. showed concern for student learning 

9. was well prepared for class 

10. presented lectures that were well-organized 

1 1 . presented lectures that were relevant 

12. provided vague answers to students’ questions (Reversed Scored) 

13. maintained control of the classroom 

14. used examples effectively to explain concepts 

15. used a wide variety of teaching aids 

16. assigned reading materials that were appropriate for the course (i.e., textbooks, articles, 
etc.) 

17. did not explain standards of excellent performance and poor performance on tests and 
papers (Reversed Scored) 

18. graded assignments, papers, tests, etc. inconsistently (Reversed Scored) 

19. provided adequate feedback on assignments and exams 

20. designed assignment procedures (tests, papers, etc.) which allowed me to demonstrate 
what I had learned 

21. presented material in a reasonable pace 

22. stayed on task well 

23. spoke at an appropriate speed for note-taking and questions 

24. used clear verbal communication 

Never (1) Seldom (2) Sometimes (3) Often (4) Always (5) NA (6) 



O 

ERIC 
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Section 3 - Rate Yourself 
Please rate yourself on. .. 

1 . your interest in taking this course before you enrolled 

2. the amount of effort this course required 

3. the extent to which you considered this a challenging course 

4. the extent to which you were motivated by the instructor to do high quality work 

5. the extent to which you struggled to understand the concepts that were presented in class 

Very Low (1) Low (2) Medium (3) High (4) 

Very High (5) 
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